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Abstract 

In an effort to reform a teacher edu¬ 
cation program by strengthening 
content-area preparation and adding 
opportunities to practice by extend¬ 
ing the time for student teaching, 
Arizona State University’s Mary Lou 
Fulton Teachers College eliminated a 
group of teacher education courses, 
including the standalone educational 
technology course. Educational tech¬ 
nology faculty members were charged 
with developing an alternative ap¬ 
proach of infusing technology into 
methods courses. Our first step was 
to conduct this benchmarking study 
of the standalone course to determine 
the successful lessons and practices 
that should be incorporated into the 
new program design. Results from 
analysis of pre- and post-course sur¬ 
vey results and focus-group data in¬ 
dicated that candidates’ confidence 
and TPACK scores increased in the 
standalone course. We will share 
benchmarks that arose from the study 
with program developers for adoption 
or adaptation to the new technology- 
infused courses where appropriate. 
Findings may also be useful to other 
teacher credentialing institutions that 
are changing to a technology-infused 
instructional approach. (Keywords: 
technology integration, TPACK, 
benchmarking, technology course, 
preservice teacher education) 


T he Mary Lou Fulton Teachers 

College is transforming its teacher 
certification program. Because it 
is one of the largest teacher credential¬ 


ing institutions in the United States, the 
actions taken with respect to technology 
integration in our preparation program 
may have implications for other prepara¬ 
tion programs. The change was triggered 
by the need to include more content- 
knowledge coursework and to require 
two full semesters of student teaching 
rather than one. To make room for these 
two new requirements, we eliminated 
a course that was once required, TEL 
313 Educational—Technology across 
the Curriculum, from the program. 
Educational technology instructors 
were charged with determining how 
candidates would learn to integrate 
technology in their lessons and teaching 
experiences without the foundational 
technology course. 

This change in requirements was 
consistent with concerns that standalone 
technology courses are ineffective in 
providing teacher education candidates 
with appropriate preparation to suc¬ 
cessfully integrate technology into their 
instruction (Bielefeldt, 2001; Moursund 
& Bielefeldt, 1999). Others have also 
written about the value of integrating 
technology into methods and content 
courses to foster technology skills more 
strongly connected to use in PK-12 in¬ 
struction and cognitive development of 
candidates (Pierson & Thompson, 2005; 
Shapely, Benner, Pieper, Way, Snider, 

& Gershner, 2003; Tonduer, van Braak, 
Sang, Voogt, Fisser, & Ottenbreit-Left- 
wich, 2012). Nevertheless, standalone 
courses have continued to be a critical 
element in initial teacher preparation 
programs (Gronseth et al., 2010; Kleiner, 
Thomas, & Lewis, 2007). Generally, 
these standalone courses provided an 


introduction to technology integration, 
and they were reinforced with technol¬ 
ogy integration in methods coursework 
(Gronseth et al., 2010; Kleiner et al., 
2007; Wang, 2006). In one study that 
was conducted to directly compare a 
standalone course with a course that 
integrated technology, Anderson and 
Borthwick (2002) found there were 
greater gains in several areas from pre- 
to post-test assessment for the group in 
the standalone course. 

The Current Standalone Educational 
Technology Course 

The teacher preparation program in¬ 
cludes 2 years of general studies and 2 
years of teacher preparation courses. All 
candidates took the required educational 
technology course, TEL 313, during the 
first semester of their junior year, and it 
was one of the first courses candidates 
took upon admission to the program. 
The course addressed issues with respect 
to helping students (a) learn how to 
use technology tools for teaching and 
learning, (b) stay updated and adopt 
a mindset of being innovative with 
technology, (c) apply ethical principles 
of using technology with their students, 
(d) attain experiences with a variety of 
technologies and their use in classroom 
settings, and (e) integrate technology 
into teaching standards-based content. 
Further, this course content was re¬ 
flected in what we asked students to do 
in major assignments. 

Candidates experienced a variety of 
active approaches, most of which both 
modeled and taught technology integra¬ 
tion, including lecture, discussion, dem¬ 
onstration, hands-on activities, project 
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presentations, and constructivist-based 
approaches. The emphasis was on devel¬ 
oping candidates’ identity as technology¬ 
using instructors. Course goals included 
participating in collaborative work 
through Web-based digital tools, design¬ 
ing digital age learning experiences, and 
participating in professional develop¬ 
ment and leadership activities including 
practice teaching. 

The following major projects ad¬ 
dressed the curriculum goals and were 
aligned with the National Education 
Technology Standards for Teachers 
(NETS»T) and Students (NETS*S; ISTE, 
2011 ): 

• Digital video storytelling: Create a 
video for teaching and learning 
through determining a worthwhile 
need, script-writing, media capture, 
editing, and sharing. 

• Mini-teach: Practice teaching using 
the TPACK framework. 

• Technology integration project: Plan, 
implement, and evaluate a project- 
based learning unit that integrates 
technology. 

A strand of curriculum address¬ 
ing digital citizenship was integrated 
throughout all of the above projects, 
with attention given to digital footprint, 
copyright and fair use, digital equity, 
adherence to policy (acceptable use), 
and other ethical concerns. 

Preparing for Change 

Although accreditation processes have 
provided one level of scrutiny to deter¬ 
mine whether programs meet required 
accreditation and preparation standards, 
including the NETS*'! (ISTE, 2011), 
true indicators of success would require 
establishing the same level of curriculum 
and candidate readiness to teach with 
technology in the reconfigured courses 
as we had in the standalone course. We 
realized a large part of this effort would 
be in proceeding systematically. 

We made two initial decisions to 
lead the change initiative. One was to 
hire a technology infusion and profes¬ 
sional development coordinator; the 
second was to designate two teacher 
education methods courses as “technol¬ 


ogy intensive.” Based on the technology 
integration results for these two courses, 
another set of courses could then be 
redesigned to be tech intensive as well. 
By the end of a 5-year period, all courses 
would be infused with appropriate 
technology integration components. By 
addressing the technology integration 
curriculum across an entire program 
instead of a single standalone course, 
teaching preservice teachers how to use 
technology would be conducted within 
the context of a content-rich environ¬ 
ment, and educational technology 
experts in the college could support the 
redevelopment of new syllabi and signa¬ 
ture assignments. 

On the one hand, it is the vision of 
technology integration advocates that 
the art of teaching with technology be 
integrated into the content of a pro¬ 
gram, rather than taught in isolation as 
a standalone course. On the other hand, 
the quality of the experience in such a 
dynamic field as technology may be best 
addressed by educational technology ex¬ 
perts, whose primary responsibility is to 
keep up with the technological advance¬ 
ments and address ongoing changes to 
course content and teaching techniques. 
Instructors who teach science, social 
studies, mathematics, and language arts 
methods courses may not be experts in 
teaching about technology integration; 
even if content instructors teach with 
technology, they may not be capable of 
teaching preservice candidates how to 
teach in PK-12 classrooms with technol¬ 
ogy. We realized that we might need to 
sacrifice some control over the owner¬ 
ship of teaching educational technology 
to advance the endeavor of integrating 
technology in a more thorough and 
programmatic way. 

Internal Benchmarking 

Benchmarking has been a practice com¬ 
monly used in business to discover suc¬ 
cessful lessons and practices that can be 
applied to create change. In benchmark¬ 
ing, organizations “learn from many dif¬ 
ferent sources and then creatively adapt 
or imitate those lessons and practices 
that will advance their performance” 
(Watson, 2009, p. xi). Benchmarking 


has supported organizations to innovate 
in ways that embrace the full circum¬ 
stance of their work, including customer 
need and situation. In benchmarking, 
organizations have conducted systematic 
observations and participated in rigor¬ 
ous conversations, which tend to create 
consensus on direction and alignment to 
the needs (Watson, 2009). 

Before the rollout of a completely 
new program structure, we had one 
more semester in which the standalone 
course was being taught. To strategically 
move from one program structure to 
another without losing the elements of 
the old program that were successful, we 
conducted an internal benchmarking 
inquiry (Global Benchmarking Network, 
n.d.). This benchmarking procedure 
allowed us to retain those successful 
lessons and best practices of our current 
program that were appropriate so that 
we could apply them to the integrated 
model, which was scheduled to begin 
the following semester. 

A benchmark process takes place in 
three stages: 

1. Determine best practices: Analyze 
product or performance 

2. Identify enablers: Identify factors as¬ 
sociated with the successful product 
or performance 

3. Lead change: Apply new learning 
to the culture of the organization 
(Watson, 2009) 

This study addressed the first stage of 
benchmarking, as we felt identification 
of successful lessons and practices of 
the current standalone program would 
support our program developers as 
they sought to amplify the effects of the 
integrated model to ensure outcomes 
that equaled or exceeded those of the 
standalone course. 

We believed our process of identify¬ 
ing successful lessons and practices 
from the standalone course was worth 
sharing, as this model of providing 
educational technology to preservice 
teachers is commonly used by other uni¬ 
versities. We realized that documenting 
our pathway to change might be useful 
to others following our lead, to adopt an 
integrated programmatic approach for 
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Figure 1. Technological Pedagogical Content Knowledge. (Adapted from http://tpack.org/ website and used by permission.) 


supporting preservice teachers who are 
learning to teach with technology. 

Framing Teaching with Technology: TPACK 

As educational technology instructors, 
we viewed the move from an isolated 
educational technology course in the 
first semester of our preservice pro¬ 
gram to infusion of technology into 
two tech-intensive courses through 
an educational framework known 
as Technological Pedagogical Con¬ 
tent Knowledge, commonly known 
as TPACK (Koehler & Mishra, 2008, 
2009; Mishra & Koehler, 2006; see also 
Pierson, 1999). The use of the TPACK 
frame provided a focus on integra¬ 
tion and offered affordances for the 
value-added features of technology. It 
encompassed aspects of teaching with 
technology that are important—name¬ 
ly the combination of technology with 
content-rich experiences and peda¬ 
gogical approaches that make good 


use of the possibilities only technology 
can provide. Pierson (1999) proposed 
a forerunner of TPACK, a theoreti¬ 
cal model of technology integration 
based on her thoughtful synthesis that 
combined technological knowledge 
with Shulman’s (1986) pedagogical 
content knowledge framework to form 
“technological-pedagogical-content 
knowledge” (p. 224). TPACK is fitting 
for our preservice program, given this 
framework relies on the equal repre¬ 
sentation of the three important and 
interrelated perspectives when devel¬ 
oping curriculum, instructional prac¬ 
tices, and accountability measures (see 
Figure 1). Details about the TPACK 
model are also provided in an article 
by Koehler and Mishra (2009). As the 
figure indicates, three types of knowl¬ 
edge required to effectively integrate 
technology are content knowledge 
(CK), pedagogical knowledge (PK), 
and technological knowledge (TK). 


Importantly, the areas that overlap 
in the figure reflect combinations of 
these different types of knowledge that 
are critical to technology integration. 
Specifically, the intersection of PK and 
CK represent a new type of knowl¬ 
edge, pedagogical content knowledge 
(PCK). Similar two-way interactions 
of TK with CK denote technologi¬ 
cal content knowledge (TCK), and 
TK with PK signify technological 
pedagogical knowledge (TPK). Finally, 
TPACK embodies the combination of 
all three types of knowledge, which is 
considered to be essential to effectively 
integrating technology into classroom 
instruction. 

The TPACK framework was origi¬ 
nally developed after 5 years of teacher 
professional development and fac¬ 
ulty development in higher education 
(Mishra & Koehler, 2006). Due to its 
generalizability to a broad range of 
developmental levels and situational 
contexts, in a mere 6 years the frame¬ 
work has been adopted as an appropri¬ 
ate lens for PK-12 students, college- 
level instructors, inservice teachers, and 
preservice candidates. 

Drawing on the TPACK work, Niess 
and her colleagues (Niess, 2011; Niess 
et al., 2009) explored the developmental 
stages of teachers who sought to use 
technology with their students. They 
suggested there were five stages: 

1. Recognizing: Understanding the 
alignment of technology to content 

2. Accepting: Having a favorable at¬ 
titude toward teaching content with 
technology 

3. Exploring: Integrating technology 
by thinking about and designing 
student-directed activities 

4. Adapting: Adopting teaching of con¬ 
tent with appropriate technology 

5. Advancing: Making revisions to cur¬ 
riculum based on results of teaching 
with technology 

The TPACK framework and Niess’ 
developmental model helped us view 
the integrated program model for ad¬ 
dressing educational technology with 
our preservice teachers as a develop¬ 
mental undertaking that may require a 
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programmatic perspective for the cur¬ 
riculum associated with teaching with 
technology, rather than a scattering 
of experiences across content courses. 
Such a developmental perspective was 
useful in conducting the benchmark¬ 
ing work in the current study and 
will be useful in the research going 
forward. Finally, perhaps some of our 
findings will be helpful to other uni¬ 
versities that have common structures 
and needs. 

Research Questions 

Consistent with benchmarking tech¬ 
niques, this step of the change process 
was meant to reveal how the current 
program was meeting the program¬ 
matic goals of preparing preservice 
teachers to teach with technology 
and to identify some of the associated 
successful lessons and practices that 
contribute to those accomplishments. 
The research questions that drove this 
inquiry included: 

. RQ1 : How and to what extent does the 
standalone course prepare candidates 
to integrate technology in teaching 
and learning? 

• RQ2: What benchmarks from the 
standalone course would be useful for 
program developers of the integrated 
model? 

Method 

The Method section provides details 
about the participants, instruments, data 
sources, data collection, and data-analy- 
ses procedures. 

Participants 

Participants included 110 candidates 
from five sections of the standalone 
course who completed both the pre- and 
post-test questionnaire. These candi¬ 
dates were juniors who were in their first 
semester in the teacher preparation pro¬ 
gram. This group of candidates included 
80 females and 30 males. The mean age 
of the participants was 24.68 years, with 
a standard deviation of 7.55 years. With 
respect to ethnicity, 80.9% were Cauca¬ 
sian, 3.6% were African-American, 12.7% 
were Hispanic, 1.8% were Asian or Pacific 
Islander, and 0.9% were Native American. 


Instruments, Data Sources, and Procedures 

Pathway Teacher Questionnaire. The 

Pathway Teacher Questionnaire (PTQ) 
was used to measure candidates’ (a) atti¬ 
tudes about the usefulness of technology 
tools, (b) dispositions toward teaching 
with technology, (c) confidence in using 
technology, and (d) technology, peda¬ 
gogy, and content knowledge, as well as 
combinations of these latter variables 
(Strudler, Schrader, & Asay, 2011). The 
PTQ was selected because it was closely 
aligned to our need to understand can¬ 
didates’ perceptions about their ability to 
become technology-using educators and 
was developed with the TPACK frame¬ 
work in mind (Koehler & Mishra, 2008, 
2009; Mishra & Koehler, 2006). It should 
be noted that the PTQ was built on the 
earlier work of Schmidt and colleagues, 
who developed the instrument to survey 
the TPACK knowledge of preservice 
candidates (Schmidt, Baran, Thomp¬ 
son, Mishra, Koehler, & Shinn, 2009). 
The questionnaire was administered to 
candidates taking the standalone course 
during the first and last weeks of a 15- 
week course. 

The PTQ consists of nine subscales. 
Three of the subscales are composed 
of 15 items each that are used to assess 
candidates’ (a) attitudes about the use¬ 
fulness of technology tools, (b) disposi¬ 
tions toward teaching with technology, 
and (c) confidence in using technology. 
An example item from the usefulness 
of technology is: “Please indicate how 
useful you find ... word processing 
software (e.g., Word).” For disposi¬ 
tions toward teaching with technology, 
an illustrative item is: “Technology 
can promote deep understanding.” 

An example item from the confidence 
scale is: “Please indicate your level of 
confidence in performing ... locate 
information online.” In addition to 
those subscales, six subscales with a 
total of 48 items were used to measure 
various components of TPACK, includ¬ 
ing 7 items that assess technological 
knowledge (TK); 15 items that measure 
content knowledge (CK); 7 items that 
evaluate pedagogical knowledge (PK); 5 
items that measure pedagogical content 


knowledge (PCK); 5 items that tap 
technological pedagogical knowledge 
(TPK); and 9 items that assess techno¬ 
logical pedagogical content knowledge 
(TPACK). To illustrate the nature of 
these items, an example from the PK 
scale is: “I can adapt my teaching style 
to different learners.” An example from 
the TPACK scale is: “I can choose tech¬ 
nologies that enhance the content for 
a lesson.” Two additional items asked 
respondents to indicate the extent to 
which they observed modeling that 
appropriately combined technology, 
pedagogy, and content knowledge 
(TPACK) in their education courses 
and in noneducation courses. The TPQ 
instrument has been shown to be valid 
and reliable (Strudler et al., 2011), and 
results of reliability analyses are pre¬ 
sented below for data from the current 
study. The survey is available, with au¬ 
thors’ permission, at http://education. 
asu.edu/documents/iTeachSurvey.pdf. 

Focus-group conversations. We used 
qualitative data to address the “how” in 
RQ1: How and to what extent does the 
standalone technology course prepare 
candidates to integrate technology in 
teaching and learning? Qualitative data 
came from focus groups of candidates in 
five different sections of the standalone 
course, from three instructors’ classes. 
We selected candidates based on a com¬ 
bination of their interest in participating 
and instructor nomination. Although fo¬ 
cus groups were comprised of candidates 
from the same section only, candidates 
retained anonymity with a participant 
number that prefaced each comment for 
ease of transcription. Candidates were 
assured their comments and recordings 
would not be made available to their 
instructors based on formal Institutional 
Review Board agreements. 

Each focus group consisted of four 
to eight candidates and was facilitated 
by educational technology experts in 
the college who were not the candidates’ 
instructors. Facilitators asked candidates 
to be sure their perspective to each ques¬ 
tion was addressed during the conver¬ 
sation, either by them or by another 
candidate. Facilitators used probing 
techniques to assure comprehensive 
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and in-depth responses. The questions 
candidates were asked included: 

1. How prepared are you to teach stu¬ 
dents to use technology to accom¬ 
plish content standards? 

2. What factors account for your level 
of preparation? 

3. What would better prepare you? 

4. Do you think you are representative 
of other candidates? 

5. How important is it to teach students 
to be critical viewers of digital 
media? 

6. How important is it to teach students 
to use technology to problem solve 
and become critical thinkers? 

7. Does it really matter whether stu¬ 
dents document sources for their 
work and work to summarize rather 
than plagiarize the work of others? 

8. Provide an example of how you 
would teach a lesson with student 
use of technology What would the 
students learn? Why would this ap¬ 
proach be better than an approach 
without technology? 

9. What tech integration have you seen 
modeled in your teacher education 
courses? What technologies? What 
activities? What assignments? 

Analysis of focus-group conversa¬ 
tions. We digitally recorded and ar¬ 
chived all focus-group conversations 
in a secured space. One of the re¬ 
searchers then reviewed the recordings 
to become immersed and to get an 
overall sense of the data. Analysis did 
not begin until the researcher gained 
an understanding of the dataset in its 
entirety (Esterberg, 2002). 

Next, following a method described 
by Miles and Huberman (1994), the 
researcher conducted an initial coding 
process involving creating word/phrase 
labels for key concepts related to seeking 
benchmarks, the goal of the study. The 
concepts met the researcher’s sense of 
being plausible, following Miles and 
Huberman’s suggestion that “plausibil¬ 
ity ... is a sort of pointer, drawing the 
analysts attention to a conclusion that 
looks reasonable and sensible on the face 
of it....” (p. 246). 


Subsequently, the researcher com¬ 
pleted the process of inductively forming 
categories, whereby she clustered the ini¬ 
tial benchmark concepts and relabeled 
specific resulting themes to exemplify 
the attributes that were most relevant to 
each theme—in this case, each bench¬ 
mark (Corbin & Strauss, 2008). Next, 
the researcher completed another pass of 
the data to identify cases where student 
remarks most closely exemplified the 
attributes of each benchmark theme. In 
sum, because the researcher returned 
to the interview data often, worked 
systemically and reflectively, and sought 
confirming and disconfirming evidence 
throughout the process, we believe 
our findings are authentic and credible 
(Miles & Huberman, 1994). 

Results 

This section presents results of the quan¬ 
titative data as well as findings based on 
the qualitative data. 

Quantitative Results 

Analysis of Pathway Teacher Question¬ 
naire. With respect to the current study, 
we obtained the following results for 
the quantitative data based on candi¬ 
dates’ responses to the PTQ. Reliability 
coefficients for pretest scores ranged 
between .82 and .98, which indicated the 
subscales demonstrated very good reli¬ 
ability. We compared pre- and post-test 
data using a repeated measures multi¬ 
variate analysis of variance (MANOVA). 
The repeated measures MANOVA was 
significant, F( 9, 85) = 8.45, p < .001, 
with partial eta-squared equal to .472. 

We conducted univariate, follow-up 
ANOVAs for each of the nine dependent 
variables. As observed in Table 1, in 
which we compared pre- and post-test 
data, there were no differences based on 
statistical tests of the pre- and post-test 
scores for usefulness of technology and 
disposition for teaching with technol¬ 
ogy, because candidates already thought 
technology was useful and were favor¬ 
ably disposed toward teaching with 
technology. On the other hand, as noted 
in Table 1, six of the remaining seven 
comparisons of pre- and post-test scores 
were significant. Confidence in using 


technology and all the TPACK model 
variables, with the exception of CK, 
increased significantly from the pre- to 
post-test administration of the instru¬ 
ment. The most noteworthy increases 
occurred for confidence in using tech¬ 
nology, TPK, and TPACK, which all 
demonstrated increases of one-half a 
point or more. PK and PCK demonstrat¬ 
ed increases of more than four-tenths of 
a point, whereas TK increased by .36 of a 
point. By comparison, CK showed a 0.10 
point increase, which was not signifi¬ 
cant. Moreover, for the various TPACK 
variables, effect sizes were all large 
effect sizes for a within-subjects design 
based on Cohen’s criteria (Olejnik & 
Algina, 2000), with the exception of CK, 
which was not significant. Cohen (1988; 
Olejnik & Algina, 2000) suggested tf 
values equal to or exceeding .01, .06, and 
.14 are considered to be small, medium, 
and large effect sizes, respectively, when 
proportion of variance accounted for is 
used as a measure of effect size. 

With respect to observing TPACK 
modeled instruction in education and 
noneducation courses, for the end-of- 
semester data, there was a statistically 
significant effect, F( 1,105) = 12.88, p < 
.001, with partial tf = .109, a medium ef¬ 
fect. The mean for modeling in the edu¬ 
cation courses was 3.95 with SD = 0.92, 
whereas the mean for the noneducation 
courses (courses offered in other colleges 
but taken by these same candidates) was 
3.66 with SD = 1.02. Overall, these data 
suggest that modeling of educational 
technology in education courses, other 
than the technology course, was higher 
than in noneducation courses. 

Results with respect to RQ1. Taken 
together, there are several noteworthy 
findings based on the quantitative data. 
First, these results suggest candidates’ 
participation in the standalone course 
increased their confidence in using 
technology. Growth in confidence was 
statistically significant and substantial, 
as demonstrated in the very large effect 
size for this variable, which showed time 
of testing accounted for more than 38% 
of the variation in scores on this mea¬ 
sure. Further, all the TPACK measures 
showed statistically significant increases, 
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Table 1 . Variables, Alpha Reliability Coefficients, Pre- and Post-Test Means and SDs, and Statistics for Subscales of the Pathway Teacher Questionnaire 


Variable 

Alpha Reliability Coefficient 

Pre- & Post-Test Means and SDs 

F-Test Statistic and df 

p Level 

Effect Size, Partial f? 2 

Usefulness of Technology 

.82 

4.31 4.35 

0.44 

<.51, 

— 



(0.56) (0.51) 

(1,107) 

NS 


Disposition for Teaching with Technology 

.98 

4.36 4.46 

1.34 

<.25, 

... 



(0.83) (0.78) 

(1,107) 

NS 


Confidence in Using Technology 

.88 

3.70 4.24 

65.80 

<.001 

.381 



(0.81) (0.66) 

(1,107) 



Technology Knowledge (TK) 

.94 

3.61 3.97 

25.88 

<.001 

.196 



(0.95) (0.87) 

(1,106) 



Content Knowledge (CK) 

.87 

3.92 4.02 

3.78 

<.06, 

— 



(0.60) (0.59) 

(1,103) 

NS 


Pedagogical Knowledge (PK) 

.89 

3.87 4.28 

33.87 

<.001 

.246 



(0.60) (0.54) 

(1,104) 



Pedagogical Content Knowledge (PCK) 

.85 

3.43 3.88 

24.81 

<.001 

.196 



(0.89) (0.69) 

(1,102) 



Technological Pedagogical Knowledge (TPK) 

.96 

3.75 4.27 

34.19 

<.001 

.247 



(0.82) (0.62) 

(1,104) 



Technological Pedagogical Content Knowledge 

.90 

3.64 4.14 

38.96 

<.001 

.280 

(TPACK) 


(0.73) (0.59) 

(1,100) 




with the exception of CK. Moreover, the 
largest gains were shown in PK, TPK, 
and TPACK. Gains in these variables 
attest to perceived changes in technol¬ 
ogy integration capabilities of these 
candidates. 

Qualitative Findings, 

Quantitative Corroboration 

In response to RQ2, the qualitative data 
resulted in the development of four 
benchmarks thought to be lessons and 
practices from the standalone course 
that might be useful in the new integra¬ 
tive approach to educational technology 
Quantitative data that confirmed or dis- 
confirmed these findings are presented 
within this section to demonstrate the 
complementarity of the quantitative and 
qualitative data (Greene, 2007). 

Benchmark 1: Technology skills. 

All candidates agreed that they were 
exposed to a wide variety of technologi¬ 
cal tools, most of which they had not 
used previously. This is typical of their 
thoughts: “I feel prepared in the sense of 
being aware of all the technology I can 
use, not so much in the sense that I can 
use them all individually.... I was satis¬ 
fied with that because it allows me to 
pick and choose the ones I would want 
to go deeper into instead of being forced 


to learn a certain number of [tools].” 
Moreover, observing their instructors 
using technology tools, leading a mini- 
teach, participating in many mini-teach- 
es, and having time to explore various 
technologies mattered. Candidates were 
grateful to have been exposed to so 
many tools, and although the focus- 
group candidates admitted that most 
candidates in their classes began the 
semester with basic technology skills, 
they concluded that they increased their 
comfort level as technology-using teach¬ 
ers. All candidates who participated in 
focus groups had a general sense that 
their comfort with technology increased. 
They felt that “before the class technol¬ 
ogy was my weakness and it was hard 
for me to learn on my own.... Now I’m 
more independent.” These findings were 
consistent with the quantitative data, 
which showed confidence for using tech¬ 
nology increased along with increases 
on the TK, PK, PCK, TPK, and TPACK 
scales from the PTQ. 

Some candidates, however, reported, 
“Some things I could have learned [on 
my own] in a fraction of the time.” 
Candidates debated about whether they 
appreciated the time spent learning 
just a few tools with depth, including 
the TPACK knowledge, or reviewing 


a broader set of tools. Overall, most 
candidates agreed that when the time 
comes for them to use a specific tool 
“with my students,” “I would know the 
technologies that would hit home with 
my students.” They said they would be 
motivated to spend the time and that 
they were adequately prepared with 
enough technology skills and know-how 
to learn almost any tool to the basic op¬ 
erational level. One insightful candidate 
reminded us that she grew up during 
the time when some key technologies, 
such as cell phones and Facebook, were 
introduced to the market. She felt she 
was more technology savvy than most 
teachers in the field. 

Benchmark 2: Technology access in 
the field. Candidates who used technol¬ 
ogy with PK-12 students were very 
motivated to learn more about teaching 
with technology because they witnessed 
its impact on students. One candidate 
reported that her internship site was a 
one-to-one computing site and her men¬ 
tor teacher was a technology coach. This 
candidates vision for technology integra¬ 
tion was expanded greatly. She reported, 
“[My mentor teachers] classroom is 95% 
paperless. It’s been really interesting to 
see how she uses it throughout the day.... 
Pretty amazing stuff.” 
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Conversely, some candidates reported 
in their field experience that they .. 
don’t really have the [desirable] ac¬ 
cess” to technology, had ill-maintained 
equipment, and their mentor teachers 
were not too concerned or interested 
in teaching with technology. They also 
indicated that in some practice teach¬ 
ing experiences, they were not afforded 
adequate access to technology tools 
and wireless Internet connection. These 
candidates felt the work that goes into 
integrating technology was not always 
worth the payoff. One candidate gave 
an example of technology limitations, 
and many others agreed: “I know in my 
[field experience] classroom, the wifi is 
just horrible. A lot of the kids have 3G 
connections that are a lot better than 
even the Internet service they have [at 
the school]. And like there’s [sic] kids 
who want to go out into the hallway so 
they can have access to the Internet. So 
when we are designing all these heav¬ 
ily technology-loaded [lessons] I have 
to wonder if I’m going to be faced with 
these situations later, and is it worth it. 
I’m not really a traditionalist, but we 
have to account for a lot of districts not 
being able to provide the kinds of things 
we want to employ.” 

Benchmark 3: Orientation of class con¬ 
tent and access to resources. Candidates 
in one focus group wanted researchers 
to know that they liked how the struc¬ 
ture of the class geared them toward 
teaching with technology because it was 
very process oriented and gave candi¬ 
dates experiences with “getting more 
familiar with the resources by doing 
[sic] [using] them.” Again, these findings 
complement the quantitative results. 

However, the content was not a 
perfect match for all candidates. Because 
candidates in the standalone educational 
technology course represented a vari¬ 
ety of grade-level and subject interests, 
some candidates felt that some class 
content was not relevant because teach¬ 
ing scenarios were mismatched to them. 
Because of this, “we don’t put the whole 
effort into [the task at hand].” As one 
candidate put it, “The things that we do 
spend a lot of time on, are not necessarily 
used in the [field experience] classroom. 


Like, we are spending four weeks on a 
digital story, which is a good technology 
to learn, but I think that four weeks could 
be spent using Smart boards or practicing 
making more lessons using technology 
... learning how to integrate it with the 
little resources that we have.” Secondary 
education candidates especially noted 
the class content was not always relevant. 
As one candidate put it, “I haven’t quite 
figured out how to incorporate technol¬ 
ogy into even half the standards. I’m a 
chemistry major, so a lot of the standards 
are based on lab activities, so it would 
be hard to incorporate [much of what I 
learned in the class].” 

This concern about content is 
consistent with the lack of change in 
the CK variable of TPACK, the only 
variable that did not change from pre- to 
post-test assessment on the TPQ. These 
candidates felt that even hypothetical 
situations presented to them during 
class meetings or mock experiences with 
a handful of real students would have 
been preferred to provide a safe first 
experience. One candidate pointed out 
that tools such as microscopes were not 
introduced in the standalone course but 
would be critical to his teaching of sci¬ 
ence. He pointed out that knowledge of 
the content his students need to learn is 
the most critical component of teaching 
with technology. 

Benchmark 4: Teaching and TPACK. 

Because this benchmark was manifested 
in terms of integrating technology 
into instruction, it draws on aspects 
of benchmarks 1, 2, and 3. Thus, for 
example, candidates must be able to 
employ their technological knowledge 
(Benchmark 1) and apply class content 
and resources (Benchmark 3) to fully 
develop technology integration skills 
they can use in teaching with technology 
(Benchmark 4). Candidates felt moder¬ 
ately prepared to teach with technology 
for a variety of reasons, including lack 
of (a) understanding of how specific 
tools might address content, (b) time to 
“play” (either during class or at home), 
(c) time to troubleshoot, and (d) com¬ 
fort with using technology with PK-12 
students due to troubleshooting. Some 
felt they lacked the kind of technology 


skills necessary to lead a technology- 
driven lesson. On the other hand, they 
felt confident in their abilities to figure 
things out, given adequate time, use of 
tutorials, online support, online teach¬ 
ing resources and idea banks, etc. One 
candidate captured the essence of con¬ 
fidence that most candidates felt when 
he said, “While we may not have had the 
time to go through and learn how to use 
all the specific features...I’d be comfort¬ 
able rolling [these tools] out with kids, 
assuming I had the time [to prepare].” 
Again, the increase in confidence exem¬ 
plified in the qualitative data is consis¬ 
tent with the increase in confidence in 
using technology demonstrated in the 
quantitative data. 

All candidates could (and most 
did) explain a technology-integrated 
unit or lesson that met the full TPACK 
framework. They were able to articulate 
the difference between teaching that 
involved technology and teaching that 
was reformed by technology to the point 
that the experience relied on it. One 
candidate gave a good example: “[Atlas 
and dictionary use] were once skills ... 
that were a building block. But now we 
just go to the Internet and in 2 seconds 
we can have like 17 definitions. So, it 
helps us, and I think that’s great. But we 
[as teachers] need to make sure we’re not 
using technology in a less than positive 
way ... not as a crutch. Where is our 
thinking going to be?” They were clear 
that the tools would continually evolve 
over time and that they would need to 
embrace this characteristic should they 
desire to help students maximize its use 
over the course of their careers. 

Candidates indicated that two of the 
signature assignment projects in the 
course were instrumental in helping 
them learn how to integrate technology. 
The Mini-Teach project, in which candi¬ 
dates were given a technology tool they 
had not explored before, and were asked 
to lead a 20-minute PK-12 teaching ex¬ 
perience to demonstrate the tool within 
the context of delivering a content-area 
lesson appropriate for school-aged 
students while using their classmates as 
“students,” was a low-risk way to practice 
teaching with technology. This project 
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appeared to be successful for candidates. 
One candidate recognized the power 
of practice when she suggested more 
of “practicing doing lessons ... teach¬ 
ing each other ... so that we have more 
hands-on practice” would be helpful for 
her future use of technology. The Tech¬ 
nology Integrated Project Plan, in which 
candidates developed a project-based 
unit that integrated technology, helped 
candidates develop real materials that 
could be used in their future teaching. 

Discussion 

In the discussion that follows, we 
examine how the benchmarks can help 
us implement technology-integration 
curriculum at high levels in the new 
integrative-technology approach. 

With respect to RQ1, “How and to 
what extent does the standalone course 
prepare candidates to integrate technol¬ 
ogy in teaching and learning?” we found 
the standalone course helped candidates 
increase their confidence for using 
technology and TPACK knowledge, and 
the effect sizes were large, with the ex¬ 
ception of CK. With respect to the lack 
of significant results for CK, note that 
candidates had not yet taken their con¬ 
tent methods courses and thus lacked a 
context for addressing this knowledge. 

Benchmark 1: Technology skills. 
Candidates need sufficient experi¬ 
ences with many technologies to foster 
confidence in learning and using new 
technologies. This benchmark suggests 
that methods courses be infused with 
opportunities to learn about several 
different digital resources and tools so 
candidates can develop a broad content- 
based technology repertoire. The intent 
is for candidates to “learn how to learn 
technology” within the context of teach¬ 
ing circumstances by developing a deep 
knowledge of one tool, as well as to gain 
awareness and basic understanding of 
other content-specific technology tools. 
As they progress through methods 
courses, we want to help them develop 
their technology skills as well as their 
ability to seek new tools so they can 
select just the right tool for the con¬ 
tent to engage their students and foster 
learning. 


Acclimating students to the world of 
teaching with technology is becoming a 
more difficult task, given the wide range 
of generations from which our students 
come. This is unique, as students in 
PK-12 are assigned curriculum by age; 
once at the university level, age does not 
prescribe curriculum. For example, the 
majority of our students are Generation 
Z students, “digital natives” who were 
born post-1990. They have “spent their 
entire lives surrounded by and using 
computers, videogames, digital music 
players, video cams, cell phones, and all 
the other toys and tools of the digital 
age” (Prensky, 2001, p. 1). They rely 
heavily on technology in their everyday 
lives, and given the opportunity and the 
resources, would take it upon themselves 
to figure out how to leverage the power 
of technology for learning (Prensky, 
2007). In the same class, we likely have 
some millennials (Generation Y) and 
Generation Xers. This scattering of stu¬ 
dents may or may not have had the same 
level of exposure to technology as our 
digital native candidates. This makes it 
more difficult to provide instruction for 
the wide range of technology skill sets 
as well as the attitudes and beliefs about 
how useful technology can be to teach¬ 
ing and learning. Further, these differ¬ 
ences establish a challenge for curricu¬ 
lum development as well as for teaching. 
We do not have an answer yet to this 
dilemma, but as we move forward with 
the idea of creating a programmatic ap¬ 
proach to developing candidates who are 
tech savvy and who teach with technol¬ 
ogy, we will be working to resolve this 
challenge through further innovation. 

Benchmark 2: Technology access in 
the field. Candidates benefit from tech¬ 
nology-rich field experiences (Bielefeldt, 
2001; Moursund & Bielefeldt, 1999). In 
our setting, finding enough classrooms 
that meet this criterion has been a diffi¬ 
cult task. Because our candidates partici¬ 
pate in field experience every semester 
of their programs, we recommend more 
diligent placement in locations with ad¬ 
equate technology access, with a teacher 
who models technology integration. On 
the other hand, we recognize that deal¬ 
ing with roadblocks is an important skill 


for technology-using teachers, and we 
support candidates developing a realistic 
sense of how they might integrate tech¬ 
nology into their instruction, no matter 
what type of access is in place. 

It seems that our challenges to 
provide students with technology-rich 
field experiences—classrooms where 
access to technology is abundant and 
where mentor teachers are capable 
and interested in exploring technology 
integration with their mentees—pose 
a significant roadblock to our work in 
helping candidates learn how to teach 
with technology. These findings are 
consistent with the conclusions drawn 
by other researchers, who indicate 
that appropriate field experiences with 
technology continue to be a formidable 
problem (Nordin, Morrow, & Davis, 
2012). Mentoring programs continue 
to be recommended as a way to provide 
support for the daily challenges that 
arise while using technology with PK-12 
students (May, 2000; O’Dwyer, Russel, 

& Bebell, 2004), especially where tools 
that are new to the field of education are 
concerned (Oigara & Wallace, 2012). Of 
significance to us, the National Council 
for the Accreditation of Teacher Educa¬ 
tion (NCATE, 2008) calls for improve¬ 
ments in the way college courses and 
field experiences address how candidates 
learn to teach with technology. 

In our programs, we continue to de¬ 
bate the role of the university in address¬ 
ing the integration of technology within 
local education settings: Should we be 
addressing technology based on the 
realities, which include limited access to 
technology in many cases, or should we 
be promoting visionary use of technol¬ 
ogy and supporting our candidates to 
be teacher-leaders who seek to push 
the limits of the systems in which they 
work? We are still deliberating about the 
most appropriate and proper balance. 
Ultimately, we recognize that the posi¬ 
tion we take will need to be feasible and 
aligned with the realities of local field 
experience settings. 

Benchmark 3: Orientation of class con¬ 
tent and access to resources in university 
courses. Candidates need sufficient 
experiences with content-related uses 
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of technology. Thus, infusing technol¬ 
ogy into methods courses provides an 
ideal environment to expose candidates 
to digital resources and tools that are 
aligned with grade levels and subject ar¬ 
eas. Our candidates were clear that when 
content and tools are taught in tandem, 
they experienced how powerful technol¬ 
ogy can be for teaching and learning. 

Teachers’ identities, beliefs, and 
perceptions of efficacy generally become 
firmly rooted during preservice course- 
work and field experiences, and fre¬ 
quently resist change (Tschannen-Moran, 
Woolfolk Hoy, & Hoy, 1998). Thus, it is 
incumbent on those preparing future 
teachers to ensure the appropriate content 
and tools are readily available as these 
identities and perceptions about efficacy 
are being shaped. Moreover, teacher 
preparation programs that are directly 
and intentionally providing candidates 
with the most current technology prac¬ 
tices and which embrace new technology 
innovations have greater potential to fos¬ 
ter candidates’ preparation for the use of 
technology (Boyd, Grossman, Lankford, 
Loeb, Wyckoff, et al., 2008). Nevertheless, 
ensuring that knowledgeable, content- 
focused faculty also embrace and remain 
current on the newest technologies and 
resources will be an ongoing need as 
technology-infused courses become the 
primary mode through which candidates 
learn to integrate technology into their 
instruction. 

Benchmark 4: Teaching and TPACK. Our 

qualitative data revealed that candidates’ 
confidence in teaching using the TPACK 
framework increased, largely based on 
their experiences teaching to their peers. 
Thus, we suggest candidates need similar 
teaching experiences in methods courses 
that fully exploit the TPACK framework, 
make them feel safe, promote taking large 
risks, and prioritize innovative uses of 
technology to address content. 

These interpretations are consistent 
with studies of preservice teachers who 
are digital natives, which suggest these 
candidates (a) are everyday users of 
quick-to-learn new technologies, (b) 
are very adept with social networking 
tools, and (c) generally use technol¬ 
ogy for personal reasons (Lei, 2009). 


But, of significance to us, they “lack the 
knowledge, skills, and experiences to 
integrate technology into classrooms 
to help them teach and to help their 
students learn, even though they fully 
recognize the importance of doing so” 
(Lei, 2009, p. 92). 

With respect to our ability to guide 
candidates to teach with technology, 
it seems that education as a whole can 
be viewed as an “early adopter” system 
(Rogers, 2002), at least in cases where we 
follow the lead of innovators. In other 
words, we are very good at adapting 
and sometimes adopting tools that were 
created for other reasons, professional 
or social (e.g., Edmodo is modeled after 
the familiar interface of Facebook but 
used to meet educational collabora¬ 
tion needs). Perhaps this adoption (or 
copy-cat) mindset is necessary, given the 
economic barriers of product develop¬ 
ment; but, as a result, we have not fully 
embraced our own creativity as educa¬ 
tors. In fact, Mishra, one of the origina¬ 
tors of TPACK, points out that bringing 
together pedagogy, content, and technol¬ 
ogy in an original manner to produce the 
kind of “learning that breaks disciplinary 
boundaries to cross-pollinate ideas, and 
thus help students become creative diver¬ 
gent thinkers” (Mishra & The Deep-Play 
Group, 2012, p. 16) may be the required 
big leap programs need to make to attain 
the next level of innovation. 

Advancing TPACK to a Programmatic 
Level for Technology Integration 

If TPACK is a developmental process, 
as Niess and colleagues suggest (Niess, 
2011; Niess et al, 2009), then is the 
one-semester educational technology 
curriculum with a traditional stand¬ 
alone course adequate to foster teaching 
using technology integration? Should 
integrating technology be purposefully 
developed throughout the program 
with a scope and sequence that is woven 
across all learning events, including field 
experiences? In any event, it will be criti¬ 
cal for instructors of methods courses to 
carefully coordinate this curriculum to 
teach to candidates’ learning trajectories, 
based on the semester in which their 
course resides. 


The mandated change from the 
standalone course to the programmatic 
approach to technology integration 
provides a one-time opportunity to 
make use of the successful lessons and 
practices obtained in our benchmark¬ 
ing work. Although we gained insights 
through this process, we also realize 
there are aspects of the technology- 
infused, programmatic approach that 
warrant thoughtful consideration as we 
move forward with this effort. 

For example, this change in who 
teaches the technology-integration 
curriculum poses an additional prob¬ 
lem, as the instructors teaching content 
methods courses are first and foremost 
experts in their content. That being said, 
they may or may not be experts in teach¬ 
ing (that content) with technology. With 
the dynamic nature of technology, this 
poses an interesting dilemma to the edu¬ 
cational technology instructors about 
how to keep content courses updated 
with technology and how to promote the 
field of educational technology creatively 
and innovatively as we see it. 

Another concern is related to field 
experiences and technology. Because 
context is an important part of the 
TPACK framework (Mishra & Koehler, 
2006), the amount and type of available 
technology in field experiences mat¬ 
ters. We realize access to technology 
is varied in the local settings, and so 
are expectations related to integrating 
technology. Because candidates will be 
experiencing curriculum that integrates 
technology beyond a single semester, 
they will be able to teach with technol¬ 
ogy in multiple school settings and with 
a variety of mentoring experiences. This 
will help them realize the possibilities, 
roadblocks, and facilitating aspects, 
including technology and personnel 
resources, that may be required. We 
expect that candidates will learn to ac¬ 
climate quickly to site cultures without 
jeopardizing their vision and interest in 
teaching with technology. Additionally, 
we expect candidates will continue to 
develop their teacher-leader qualities 
by seeking new and innovative meth¬ 
ods, given the available resources in 
the field, and that they will become 
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advocates of change by promoting 
the integration of technology by their 
peers. Of course, this also means our 
field placement staff will be seeking 
to find more technology-rich sites for 
field experiences every semester, rather 
than just the semester required for the 
standalone course. 

Conclusion 

Although responding to change can be 
challenging, it affords wonderful op¬ 
portunities to be innovative and creative, 
as well. That is the case with our situa¬ 
tion, when we were asked to eliminate a 
standalone technology class and replace 
it with technology-infused courses 
in our teacher preparation programs. 
Naturally, our efforts are in the initial 
stages of the process, but we believe the 
benchmarking process will allow us to 
incorporate the best ideas, activities, 
and instructional processes from the 
standalone course into the to-be-devel- 
oped technology-infused courses that 
will be included throughout candidates’ 
preparation. 

We anticipate that this phase of 
reporting on our innovative approach to 
addressing the development of preser¬ 
vice teachers’ abilities to teach with tech¬ 
nology will result in two very important 
contributions. First, we will share the 
benchmarks with program developers 
in our college so that they can make use 
of the nuggets of success currently in 
place in the standalone course. Second, 
we believe our findings will contribute 
understanding and insights to others 
who provide technology instruction to 
preservice candidates or to others who 
might follow our lead by reforming to 
a program-wide curriculum for educa¬ 
tional technology. 

Times are exciting as we anticipate 
the development and implementation of 
our vision of truly integrating tech¬ 
nology into content. We plan to fully 
document the next steps, including the 
process of change as well as the sum- 
mative results about how the TPACK 
framework assists instructors to meet 
the needs of preservice teachers to teach 
with technology using the integrated 
model. We shall see. 
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